{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Example Seldon Core Deployments using Helm\n",
    "<img src=\"images/deploy-graph.png\" alt=\"predictor with canary\" title=\"ml graph\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prerequistes\n",
    "You will need\n",
    " - [Git clone of Seldon Core](https://github.com/SeldonIO/seldon-core)\n",
    " - A running Kubernetes cluster with kubectl authenticated\n",
    " - [python grpc tools](https://grpc.io/docs/quickstart/python.html)\n",
    " - [Helm client](https://helm.sh/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating a Kubernetes Cluster\n",
    "\n",
    "Follow the [Kubernetes documentation to create a cluster](https://kubernetes.io/docs/setup/).\n",
    "\n",
    "Once created ensure ```kubectl``` is authenticated against the running cluster."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "namespace/seldon created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl create namespace seldon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Context \"minikube\" modified.\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl config set-context $(kubectl config current-context) --namespace=seldon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "clusterrolebinding.rbac.authorization.k8s.io/kube-system-cluster-admin created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl create clusterrolebinding kube-system-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Install Helm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "serviceaccount/tiller created\n",
      "clusterrolebinding.rbac.authorization.k8s.io/tiller created\n",
      "$HELM_HOME has been configured at /home/clive/.helm.\n",
      "\n",
      "Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.\n",
      "\n",
      "Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.\n",
      "To prevent this, run `helm init` with the --tiller-tls-verify flag.\n",
      "For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation\n",
      "Happy Helming!\n"
     ]
    }
   ],
   "source": [
    "!kubectl -n kube-system create sa tiller\n",
    "!kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller\n",
    "!helm init --service-account tiller"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waiting for deployment \"tiller-deploy\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"tiller-deploy\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/tiller-deploy -n kube-system"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Start seldon-core"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   seldon-core-crd\n",
      "LAST DEPLOYED: Fri Oct 19 18:30:11 2018\n",
      "NAMESPACE: seldon\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1/ConfigMap\n",
      "NAME                     DATA  AGE\n",
      "seldon-spartakus-config  3     0s\n",
      "\n",
      "==> v1beta1/CustomResourceDefinition\n",
      "NAME                                         AGE\n",
      "seldondeployments.machinelearning.seldon.io  0s\n",
      "\n",
      "==> v1beta1/Deployment\n",
      "NAME                        DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE\n",
      "seldon-spartakus-volunteer  1        0        0           0          0s\n",
      "\n",
      "==> v1/ServiceAccount\n",
      "NAME                        SECRETS  AGE\n",
      "seldon-spartakus-volunteer  1        0s\n",
      "\n",
      "==> v1beta1/ClusterRole\n",
      "NAME                        AGE\n",
      "seldon-spartakus-volunteer  0s\n",
      "\n",
      "==> v1beta1/ClusterRoleBinding\n",
      "NAME                        AGE\n",
      "seldon-spartakus-volunteer  0s\n",
      "\n",
      "\n",
      "NOTES:\n",
      "NOTES: TODO\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install ../../helm-charts/seldon-core-crd --name seldon-core-crd --set usage_metrics.enabled=true"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   seldon-core\n",
      "LAST DEPLOYED: Fri Oct 19 18:33:43 2018\n",
      "NAMESPACE: seldon\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1/ServiceAccount\n",
      "NAME    SECRETS  AGE\n",
      "seldon  1        0s\n",
      "\n",
      "==> v1beta1/ClusterRole\n",
      "NAME        AGE\n",
      "seldon-crd  0s\n",
      "\n",
      "==> v1/ClusterRoleBinding\n",
      "NAME    AGE\n",
      "seldon  0s\n",
      "\n",
      "==> v1beta1/Role\n",
      "NAME          AGE\n",
      "seldon-local  0s\n",
      "ambassador    0s\n",
      "\n",
      "==> v1beta1/RoleBinding\n",
      "NAME        AGE\n",
      "ambassador  0s\n",
      "\n",
      "==> v1beta1/Deployment\n",
      "NAME                                DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE\n",
      "seldon-core-ambassador              1        1        1           0          0s\n",
      "seldon-core-seldon-apiserver        1        1        1           0          0s\n",
      "seldon-core-seldon-cluster-manager  1        1        1           0          0s\n",
      "seldon-core-redis                   1        1        1           0          0s\n",
      "\n",
      "==> v1/RoleBinding\n",
      "NAME    AGE\n",
      "seldon  0s\n",
      "\n",
      "==> v1/Service\n",
      "NAME                          TYPE       CLUSTER-IP      EXTERNAL-IP  PORT(S)                        AGE\n",
      "seldon-core-ambassador        NodePort   10.106.219.166  <none>       8080:30612/TCP                 0s\n",
      "seldon-core-ambassador-admin  NodePort   10.105.30.215   <none>       8877:30711/TCP                 0s\n",
      "seldon-core-seldon-apiserver  NodePort   10.99.110.157   <none>       8080:32220/TCP,5000:30494/TCP  0s\n",
      "seldon-core-redis             ClusterIP  10.107.202.79   <none>       6379/TCP                       0s\n",
      "\n",
      "==> v1/Pod(related)\n",
      "NAME                                                 READY  STATUS             RESTARTS  AGE\n",
      "seldon-core-ambassador-778c58bf5d-2fjzn              0/2    ContainerCreating  0         0s\n",
      "seldon-core-ambassador-778c58bf5d-98p5g              0/2    Terminating        0         3m\n",
      "seldon-core-seldon-apiserver-6b8dbc978b-45c7r        1/1    Terminating        0         3m\n",
      "seldon-core-seldon-apiserver-6b8dbc978b-pfk6n        0/1    ContainerCreating  0         0s\n",
      "seldon-core-seldon-cluster-manager-596d4674fd-242ps  0/1    ContainerCreating  0         0s\n",
      "seldon-core-redis-8668565565-rtzc7                   0/1    ContainerCreating  0         0s\n",
      "\n",
      "\n",
      "NOTES:\n",
      "Thank you for installing Seldon Core.\n",
      "\n",
      "Documentation can be found at https://github.com/SeldonIO/seldon-core\n",
      "\n",
      "\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install ../../helm-charts/seldon-core --name seldon-core --namespace seldon  --set ambassador.enabled=true"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "deployment \"seldon-core-seldon-cluster-manager\" successfully rolled out\n",
      "deployment \"seldon-core-seldon-apiserver\" successfully rolled out\n",
      "Waiting for deployment \"seldon-core-ambassador\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"seldon-core-ambassador\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/seldon-core-seldon-cluster-manager\n",
    "!kubectl rollout status deploy/seldon-core-seldon-apiserver\n",
    "!kubectl rollout status deploy/seldon-core-ambassador"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set up REST and gRPC methods\n",
    "\n",
    "**Ensure you port forward ambassador and API gateway**:\n",
    "\n",
    "```\n",
    "kubectl port-forward $(kubectl get pods -n seldon -l service=ambassador -o jsonpath='{.items[0].metadata.name}') -n seldon 8003:8080\n",
    "```\n",
    "\n",
    "```\n",
    "kubectl port-forward $(kubectl get pods -n seldon -l app=seldon-apiserver-container-app -o jsonpath='{.items[0].metadata.name}') -n seldon 8002:8080\n",
    "```\n",
    "\n",
    "```\n",
    "kubectl port-forward $(kubectl get pods -n seldon -l app=seldon-apiserver-container-app -o jsonpath='{.items[0].metadata.name}') -n seldon 8004:5000\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Install gRPC modules for the prediction protos."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cp: cannot stat '../proto/prediction.proto': No such file or directory\n",
      "./proto/prediction.proto: No such file or directory\n"
     ]
    }
   ],
   "source": [
    "!cp ../proto/prediction.proto ./proto\n",
    "!python -m grpc.tools.protoc -I. --python_out=. --grpc_python_out=. ./proto/prediction.proto"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Serve Single Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   mymodel\n",
      "LAST DEPLOYED: Fri Oct 19 18:35:50 2018\n",
      "NAMESPACE: seldon\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1alpha2/SeldonDeployment\n",
      "NAME     AGE\n",
      "mymodel  0s\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install ../../helm-charts/seldon-single-model --name mymodel --set oauth.key=oauth-key --set oauth.secret=oauth-secret"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\r\n",
      "\u001b[04m\u001b[31;01m#\u001b[39;49;00m \u001b[04m\u001b[31;01mS\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mu\u001b[39;49;00m\u001b[04m\u001b[31;01mr\u001b[39;49;00m\u001b[04m\u001b[31;01mc\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01m:\u001b[39;49;00m \u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mi\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\u001b[04m\u001b[31;01mg\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01m/\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01mp\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01m/\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01m.\u001b[39;49;00m\u001b[04m\u001b[31;01mj\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\r\n",
      "{\r\n",
      "    \u001b[34;01m\"apiVersion\"\u001b[39;49;00m: \u001b[33m\"machinelearning.seldon.io/v1alpha2\"\u001b[39;49;00m,\r\n",
      "    \u001b[34;01m\"kind\"\u001b[39;49;00m: \u001b[33m\"SeldonDeployment\"\u001b[39;49;00m,\r\n",
      "    \u001b[34;01m\"metadata\"\u001b[39;49;00m: {\r\n",
      "        \u001b[34;01m\"labels\"\u001b[39;49;00m: {\r\n",
      "            \u001b[34;01m\"app\"\u001b[39;49;00m: \u001b[33m\"seldon\"\u001b[39;49;00m\r\n",
      "        },\r\n",
      "        \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"mymodel\"\u001b[39;49;00m\r\n",
      "    },\r\n",
      "    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "        \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"mymodel\"\u001b[39;49;00m,        \r\n",
      "        \u001b[34;01m\"oauth_key\"\u001b[39;49;00m: \u001b[33m\"oauth-key\"\u001b[39;49;00m,\r\n",
      "        \u001b[34;01m\"oauth_secret\"\u001b[39;49;00m: \u001b[33m\"oauth-secret\"\u001b[39;49;00m,\r\n",
      "        \u001b[34;01m\"predictors\"\u001b[39;49;00m: [\r\n",
      "            {\r\n",
      "                \u001b[34;01m\"componentSpecs\"\u001b[39;49;00m: [{\r\n",
      "                    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "                        \u001b[34;01m\"containers\"\u001b[39;49;00m: [\r\n",
      "                            {\r\n",
      "                                \u001b[34;01m\"image\"\u001b[39;49;00m: \u001b[33m\"seldonio/mock_classifier:1.0\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"imagePullPolicy\"\u001b[39;49;00m: \u001b[33m\"IfNotPresent\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"resources\"\u001b[39;49;00m: {\r\n",
      "                                    \u001b[34;01m\"requests\"\u001b[39;49;00m: {\r\n",
      "                                        \u001b[34;01m\"memory\"\u001b[39;49;00m: \u001b[33m\"1Mi\"\u001b[39;49;00m\r\n",
      "                                    }\r\n",
      "                                }\r\n",
      "                            }\r\n",
      "                        ],\r\n",
      "                        \u001b[34;01m\"terminationGracePeriodSeconds\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m\r\n",
      "                    }}\t\t\t\t   \r\n",
      "\t        ],\r\n",
      "                \u001b[34;01m\"graph\"\u001b[39;49;00m:\t\t\t \r\n",
      "\t\t{\r\n",
      "                    \u001b[34;01m\"children\"\u001b[39;49;00m: [],\r\n",
      "                    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier\"\u001b[39;49;00m,\r\n",
      "                    \u001b[34;01m\"endpoint\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\u001b[34;01m\"type\"\u001b[39;49;00m : \u001b[33m\"REST\"\u001b[39;49;00m\r\n",
      "\t\t    },\r\n",
      "                    \u001b[34;01m\"type\"\u001b[39;49;00m: \u001b[33m\"MODEL\"\u001b[39;49;00m\r\n",
      "                },\t\t\t \t\t\t \r\n",
      "                \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"mymodel\"\u001b[39;49;00m,\r\n",
      "                \u001b[34;01m\"replicas\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m,\r\n",
      "\t\t\u001b[34;01m\"labels\"\u001b[39;49;00m: {\r\n",
      "\t\t    \u001b[34;01m\"version\"\u001b[39;49;00m : \u001b[33m\"v1\"\u001b[39;49;00m\r\n",
      "\t\t}\r\n",
      "            }\r\n",
      "        ]\r\n",
      "    }\r\n",
      "}\r\n"
     ]
    }
   ],
   "source": [
    "!helm template ../../helm-charts/seldon-single-model --name mymodel --set oauth.key=oauth-key --set oauth.secret=oauth-secret | pygmentize -l json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waiting for deployment \"mymodel-mymodel-classifier-0\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymodel-mymodel-classifier-0\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymodel-mymodel-classifier-0\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymodel-mymodel-classifier-0\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"mymodel-mymodel-classifier-0\" successfully rolled out\n",
      "deployment \"mymodel-mymodel-svc-orch\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/mymodel-mymodel-classifier-0\n",
    "!kubectl rollout status deploy/mymodel-mymodel-svc-orch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.path.append(\"../../notebooks\")\n",
    "from seldon_utils import *\n",
    "API_AMBASSADOR=\"localhost:8003\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### REST Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"7892c1ce-ab4b-43b8-a836-fbf872d7dba9\",\"token_type\":\"bearer\",\"expires_in\":43199,\"scope\":\"read write\"}\n",
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"5qeb57bpbj3p82tjc0nb9v2s0q\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "    },\n",
      "    \"requestPath\": {\n",
      "      \"classifier\": \"seldonio/mock_classifier:1.0\"\n",
      "    }\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.07824180738722948]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "r = rest_request_api_gateway(\"oauth-key\",\"oauth-secret\")\n",
    "print(r.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"ikg7ft60992a1tgk5k3eceogmg\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "    },\n",
      "    \"requestPath\": {\n",
      "      \"classifier\": \"seldonio/mock_classifier:1.0\"\n",
      "    }\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.0871649338678916]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "r = rest_request_ambassador(\"mymodel\",API_AMBASSADOR)\n",
    "print(r.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### gRPC Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "meta {\n",
       "  puid: \"me6t461pjhk2gotkif3qmdbv9i\"\n",
       "  requestPath {\n",
       "    key: \"classifier\"\n",
       "    value: \"seldonio/mock_classifier:1.0\"\n",
       "  }\n",
       "}\n",
       "data {\n",
       "  names: \"proba\"\n",
       "  tensor {\n",
       "    shape: 1\n",
       "    shape: 1\n",
       "    values: 0.06833286694600807\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grpc_request_ambassador(\"mymodel\",API_AMBASSADOR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"7892c1ce-ab4b-43b8-a836-fbf872d7dba9\",\"token_type\":\"bearer\",\"expires_in\":43190,\"scope\":\"read write\"}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "meta {\n",
       "  puid: \"oi30b58mika5ismlnjktpnda8n\"\n",
       "  requestPath {\n",
       "    key: \"classifier\"\n",
       "    value: \"seldonio/mock_classifier:1.0\"\n",
       "  }\n",
       "}\n",
       "data {\n",
       "  names: \"proba\"\n",
       "  tensor {\n",
       "    shape: 1\n",
       "    shape: 1\n",
       "    values: 0.09278521594160527\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grpc_request_api_gateway(\"oauth-key\",\"oauth-secret\",grpc_endpoint=\"localhost:8004\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "release \"mymodel\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!helm delete mymodel --purge"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Serve AB Test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   myabtest\r\n",
      "LAST DEPLOYED: Fri Oct 19 18:45:42 2018\r\n",
      "NAMESPACE: seldon\r\n",
      "STATUS: DEPLOYED\r\n",
      "\r\n",
      "RESOURCES:\r\n",
      "==> v1alpha2/SeldonDeployment\r\n",
      "NAME      AGE\r\n",
      "myabtest  0s\r\n",
      "\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "!helm install ../../helm-charts/seldon-abtest --name myabtest --set oauth.key=oauth-key --set oauth.secret=oauth-secret"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\r\n",
      "\u001b[04m\u001b[31;01m#\u001b[39;49;00m \u001b[04m\u001b[31;01mS\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mu\u001b[39;49;00m\u001b[04m\u001b[31;01mr\u001b[39;49;00m\u001b[04m\u001b[31;01mc\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01m:\u001b[39;49;00m \u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01mb\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01m/\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01mp\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01m/\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01mb\u001b[39;49;00m\u001b[04m\u001b[31;01m_\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01m_\u001b[39;49;00m\u001b[34m1\u001b[39;49;00m\u001b[04m\u001b[31;01mp\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01m.\u001b[39;49;00m\u001b[04m\u001b[31;01mj\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\r\n",
      "\r\n",
      "\r\n",
      "\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\r\n",
      "\u001b[04m\u001b[31;01m#\u001b[39;49;00m \u001b[04m\u001b[31;01mS\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mu\u001b[39;49;00m\u001b[04m\u001b[31;01mr\u001b[39;49;00m\u001b[04m\u001b[31;01mc\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01m:\u001b[39;49;00m \u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01mb\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01m/\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01mp\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01m/\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01mb\u001b[39;49;00m\u001b[04m\u001b[31;01m_\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01m_\u001b[39;49;00m\u001b[34m2\u001b[39;49;00m\u001b[04m\u001b[31;01mp\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01m.\u001b[39;49;00m\u001b[04m\u001b[31;01mj\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\r\n",
      "\r\n",
      "{\r\n",
      "    \u001b[34;01m\"apiVersion\"\u001b[39;49;00m: \u001b[33m\"machinelearning.seldon.io/v1alpha2\"\u001b[39;49;00m,\r\n",
      "    \u001b[34;01m\"kind\"\u001b[39;49;00m: \u001b[33m\"SeldonDeployment\"\u001b[39;49;00m,\r\n",
      "    \u001b[34;01m\"metadata\"\u001b[39;49;00m: {\r\n",
      "\t\u001b[34;01m\"labels\"\u001b[39;49;00m: {\r\n",
      "\t    \u001b[34;01m\"app\"\u001b[39;49;00m: \u001b[33m\"seldon\"\u001b[39;49;00m\r\n",
      "\t},\r\n",
      "\t\u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"myabtest\"\u001b[39;49;00m\r\n",
      "    },\r\n",
      "    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "\t\u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"myabtest\"\u001b[39;49;00m,        \r\n",
      "        \u001b[34;01m\"oauth_key\"\u001b[39;49;00m: \u001b[33m\"oauth-key\"\u001b[39;49;00m,\r\n",
      "        \u001b[34;01m\"oauth_secret\"\u001b[39;49;00m: \u001b[33m\"oauth-secret\"\u001b[39;49;00m,\r\n",
      "\t\u001b[34;01m\"predictors\"\u001b[39;49;00m: [\r\n",
      "\t    {\r\n",
      "\t\t\u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"abtest\"\u001b[39;49;00m,\r\n",
      "\t\t\u001b[34;01m\"replicas\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m,\r\n",
      "\t\t\u001b[34;01m\"componentSpecs\"\u001b[39;49;00m: [{\r\n",
      "\t\t    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\u001b[34;01m\"containers\"\u001b[39;49;00m: [\r\n",
      "\t\t\t    {\r\n",
      "\t\t\t\t\u001b[34;01m\"image\"\u001b[39;49;00m: \u001b[33m\"seldonio/mock_classifier:1.0\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"imagePullPolicy\"\u001b[39;49;00m: \u001b[33m\"IfNotPresent\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier-1\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"resources\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t    \u001b[34;01m\"requests\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t\t\u001b[34;01m\"memory\"\u001b[39;49;00m: \u001b[33m\"1Mi\"\u001b[39;49;00m\r\n",
      "\t\t\t\t    }\r\n",
      "\t\t\t\t}\r\n",
      "\t\t\t    }],\r\n",
      "\t\t\t\u001b[34;01m\"terminationGracePeriodSeconds\"\u001b[39;49;00m: \u001b[34m20\u001b[39;49;00m\r\n",
      "\t\t    }},\r\n",
      "\t        {\r\n",
      "\t\t    \u001b[34;01m\"metadata\"\u001b[39;49;00m:{\r\n",
      "\t\t\t\u001b[34;01m\"labels\"\u001b[39;49;00m:{\r\n",
      "\t\t\t    \u001b[34;01m\"version\"\u001b[39;49;00m:\u001b[33m\"v2\"\u001b[39;49;00m\r\n",
      "\t\t\t}\r\n",
      "\t\t    },    \r\n",
      "\t\t\t\u001b[34;01m\"spec\"\u001b[39;49;00m:{\r\n",
      "\t\t\t    \u001b[34;01m\"containers\"\u001b[39;49;00m:[\r\n",
      "\t\t\t    {\r\n",
      "\t\t\t\t\u001b[34;01m\"image\"\u001b[39;49;00m: \u001b[33m\"seldonio/mock_classifier:1.0\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"imagePullPolicy\"\u001b[39;49;00m: \u001b[33m\"IfNotPresent\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier-2\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"resources\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t    \u001b[34;01m\"requests\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t\t\u001b[34;01m\"memory\"\u001b[39;49;00m: \u001b[33m\"1Mi\"\u001b[39;49;00m\r\n",
      "\t\t\t\t    }\r\n",
      "\t\t\t\t}\r\n",
      "\t\t\t    }\r\n",
      "\t\t\t],\r\n",
      "\t\t\t\u001b[34;01m\"terminationGracePeriodSeconds\"\u001b[39;49;00m: \u001b[34m20\u001b[39;49;00m\r\n",
      "\t\t\t\t   }\r\n",
      "\t\t\t\t   }],\r\n",
      "\t\t\u001b[34;01m\"graph\"\u001b[39;49;00m: {\r\n",
      "\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"myabtest\"\u001b[39;49;00m,\r\n",
      "\t\t    \u001b[34;01m\"endpoint\"\u001b[39;49;00m:{},\r\n",
      "\t\t    \u001b[34;01m\"implementation\"\u001b[39;49;00m:\u001b[33m\"RANDOM_ABTEST\"\u001b[39;49;00m,\r\n",
      "\t\t    \u001b[34;01m\"parameters\"\u001b[39;49;00m: [\r\n",
      "\t\t\t{\r\n",
      "\t\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m:\u001b[33m\"ratioA\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"value\"\u001b[39;49;00m:\u001b[33m\"0.5\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m:\u001b[33m\"FLOAT\"\u001b[39;49;00m\r\n",
      "\t\t\t}\r\n",
      "\t\t    ],\r\n",
      "\t\t    \u001b[34;01m\"children\"\u001b[39;49;00m: [\r\n",
      "\t\t\t{\r\n",
      "\t\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier-1\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"endpoint\"\u001b[39;49;00m:{\r\n",
      "\t\t\t\t\u001b[34;01m\"type\"\u001b[39;49;00m:\u001b[33m\"REST\"\u001b[39;49;00m\r\n",
      "\t\t\t    },\r\n",
      "\t\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m:\u001b[33m\"MODEL\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"children\"\u001b[39;49;00m:[]\r\n",
      "\t\t\t},\r\n",
      "\t\t\t{\r\n",
      "\t\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier-2\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"endpoint\"\u001b[39;49;00m:{\r\n",
      "\t\t\t\t\u001b[34;01m\"type\"\u001b[39;49;00m:\u001b[33m\"REST\"\u001b[39;49;00m\r\n",
      "\t\t\t    },\r\n",
      "\t\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m:\u001b[33m\"MODEL\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"children\"\u001b[39;49;00m:[]\r\n",
      "\t\t\t}   \r\n",
      "\t\t    ]\r\n",
      "\t\t}\r\n",
      "\t    }\r\n",
      "\t]\r\n",
      "    }\r\n",
      "}\t\t\r\n",
      "\t\t\r\n",
      "\r\n",
      " \r\n"
     ]
    }
   ],
   "source": [
    "!helm template ../../helm-charts/seldon-abtest --name myabtest --set oauth.key=oauth-key --set oauth.secret=oauth-secret | pygmentize -l json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "deployment \"myabtest-abtest-classifier-1-0\" successfully rolled out\n",
      "deployment \"myabtest-abtest-classifier-1-0\" successfully rolled out\n",
      "deployment \"myabtest-abtest-svc-orch\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/myabtest-abtest-classifier-1-0\n",
    "!kubectl rollout status deploy/myabtest-abtest-classifier-1-0\n",
    "!kubectl rollout status deploy/myabtest-abtest-svc-orch "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### REST Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"ifcr21qadvou58dmc76kgiu24v\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "      \"myabtest\": 1\n",
      "    },\n",
      "    \"requestPath\": {\n",
      "      \"myabtest\": \"\",\n",
      "      \"classifier-2\": \"seldonio/mock_classifier:1.0\"\n",
      "    }\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.0849806651591639]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "r = rest_request_ambassador(\"myabtest\",API_AMBASSADOR)\n",
    "print(r.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"7892c1ce-ab4b-43b8-a836-fbf872d7dba9\",\"token_type\":\"bearer\",\"expires_in\":43076,\"scope\":\"read write\"}\n",
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"chg6ps28i2hnc6oa4dptcra4ph\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "      \"myabtest\": 0\n",
      "    },\n",
      "    \"requestPath\": {\n",
      "      \"myabtest\": \"\",\n",
      "      \"classifier-1\": \"seldonio/mock_classifier:1.0\"\n",
      "    }\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.10304073265496357]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "r = rest_request_api_gateway(\"oauth-key\",\"oauth-secret\")\n",
    "print(r.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### gRPC Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "meta {\n",
       "  puid: \"mfd2023j1a3ethn5j5o9o5k2jf\"\n",
       "  routing {\n",
       "    key: \"myabtest\"\n",
       "    value: 1\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"classifier-2\"\n",
       "    value: \"seldonio/mock_classifier:1.0\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"myabtest\"\n",
       "  }\n",
       "}\n",
       "data {\n",
       "  names: \"proba\"\n",
       "  tensor {\n",
       "    shape: 1\n",
       "    shape: 1\n",
       "    values: 0.07409465499499068\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grpc_request_ambassador(\"myabtest\",API_AMBASSADOR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"7892c1ce-ab4b-43b8-a836-fbf872d7dba9\",\"token_type\":\"bearer\",\"expires_in\":43050,\"scope\":\"read write\"}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "meta {\n",
       "  puid: \"9egs54q5lotncnqknj9gbdobei\"\n",
       "  routing {\n",
       "    key: \"myabtest\"\n",
       "    value: 1\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"classifier-2\"\n",
       "    value: \"seldonio/mock_classifier:1.0\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"myabtest\"\n",
       "  }\n",
       "}\n",
       "data {\n",
       "  names: \"proba\"\n",
       "  tensor {\n",
       "    shape: 1\n",
       "    shape: 1\n",
       "    values: 0.08308652534451443\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grpc_request_api_gateway(\"oauth-key\",\"oauth-secret\",grpc_endpoint=\"localhost:8004\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "release \"myabtest\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!helm delete myabtest --purge"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Serve Multi-Armed Bandit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   mymab\r\n",
      "LAST DEPLOYED: Fri Oct 19 18:47:57 2018\r\n",
      "NAMESPACE: seldon\r\n",
      "STATUS: DEPLOYED\r\n",
      "\r\n",
      "RESOURCES:\r\n",
      "==> v1alpha2/SeldonDeployment\r\n",
      "NAME   AGE\r\n",
      "mymab  0s\r\n",
      "\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "!helm install ../../helm-charts/seldon-mab --name mymab --set oauth.key=oauth-key --set oauth.secret=oauth-secret"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\r\n",
      "\u001b[04m\u001b[31;01m#\u001b[39;49;00m \u001b[04m\u001b[31;01mS\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mu\u001b[39;49;00m\u001b[04m\u001b[31;01mr\u001b[39;49;00m\u001b[04m\u001b[31;01mc\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01m:\u001b[39;49;00m \u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01mb\u001b[39;49;00m\u001b[04m\u001b[31;01m/\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01mp\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01m/\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01mb\u001b[39;49;00m\u001b[04m\u001b[31;01m.\u001b[39;49;00m\u001b[04m\u001b[31;01mj\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\r\n",
      "{\r\n",
      "    \u001b[34;01m\"apiVersion\"\u001b[39;49;00m: \u001b[33m\"machinelearning.seldon.io/v1alpha2\"\u001b[39;49;00m,\r\n",
      "    \u001b[34;01m\"kind\"\u001b[39;49;00m: \u001b[33m\"SeldonDeployment\"\u001b[39;49;00m,\r\n",
      "    \u001b[34;01m\"metadata\"\u001b[39;49;00m: {\r\n",
      "\t\u001b[34;01m\"labels\"\u001b[39;49;00m: {\r\n",
      "\t    \u001b[34;01m\"app\"\u001b[39;49;00m: \u001b[33m\"seldon\"\u001b[39;49;00m\r\n",
      "\t},\r\n",
      "\t\u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"mymab\"\u001b[39;49;00m\r\n",
      "    },\r\n",
      "    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "\t\u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"mymab\"\u001b[39;49;00m,        \r\n",
      "        \u001b[34;01m\"oauth_key\"\u001b[39;49;00m: \u001b[33m\"oauth-key\"\u001b[39;49;00m,\r\n",
      "        \u001b[34;01m\"oauth_secret\"\u001b[39;49;00m: \u001b[33m\"oauth-secret\"\u001b[39;49;00m,\r\n",
      "\t\u001b[34;01m\"predictors\"\u001b[39;49;00m: [\r\n",
      "\t    {\r\n",
      "\t\t\u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"abtest\"\u001b[39;49;00m,\r\n",
      "\t\t\u001b[34;01m\"replicas\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m,\r\n",
      "\t\t\u001b[34;01m\"componentSpecs\"\u001b[39;49;00m: [{\r\n",
      "\t\t    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\u001b[34;01m\"containers\"\u001b[39;49;00m: [\r\n",
      "\t\t\t    {\r\n",
      "\t\t\t\t\u001b[34;01m\"image\"\u001b[39;49;00m: \u001b[33m\"seldonio/mock_classifier:1.0\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"imagePullPolicy\"\u001b[39;49;00m: \u001b[33m\"IfNotPresent\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier-1\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"resources\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t    \u001b[34;01m\"requests\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t\t\u001b[34;01m\"memory\"\u001b[39;49;00m: \u001b[33m\"1Mi\"\u001b[39;49;00m\r\n",
      "\t\t\t\t    }\r\n",
      "\t\t\t\t}\r\n",
      "\t\t\t    }],\r\n",
      "\t\t\t\u001b[34;01m\"terminationGracePeriodSeconds\"\u001b[39;49;00m: \u001b[34m20\u001b[39;49;00m\r\n",
      "\t\t    }},\r\n",
      "\t        {\r\n",
      "\t\t    \u001b[34;01m\"metadata\"\u001b[39;49;00m:{\r\n",
      "\t\t\t\u001b[34;01m\"labels\"\u001b[39;49;00m:{\r\n",
      "\t\t\t    \u001b[34;01m\"version\"\u001b[39;49;00m:\u001b[33m\"v2\"\u001b[39;49;00m\r\n",
      "\t\t\t}\r\n",
      "\t\t    },    \r\n",
      "\t\t\t\u001b[34;01m\"spec\"\u001b[39;49;00m:{\r\n",
      "\t\t\t    \u001b[34;01m\"containers\"\u001b[39;49;00m:[\r\n",
      "\t\t\t    {\r\n",
      "\t\t\t\t\u001b[34;01m\"image\"\u001b[39;49;00m: \u001b[33m\"seldonio/mock_classifier:1.0\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"imagePullPolicy\"\u001b[39;49;00m: \u001b[33m\"IfNotPresent\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier-2\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"resources\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t    \u001b[34;01m\"requests\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t\t\u001b[34;01m\"memory\"\u001b[39;49;00m: \u001b[33m\"1Mi\"\u001b[39;49;00m\r\n",
      "\t\t\t\t    }\r\n",
      "\t\t\t\t}\r\n",
      "\t\t\t    }\r\n",
      "\t\t\t],\r\n",
      "\t\t\t\u001b[34;01m\"terminationGracePeriodSeconds\"\u001b[39;49;00m: \u001b[34m20\u001b[39;49;00m\r\n",
      "\t\t\t}\r\n",
      "\t\t},\r\n",
      "\t        {\r\n",
      "\t\t    \u001b[34;01m\"spec\"\u001b[39;49;00m:{\r\n",
      "\t\t\t\u001b[34;01m\"containers\"\u001b[39;49;00m: [{\r\n",
      "\t\t\t    \u001b[34;01m\"image\"\u001b[39;49;00m: \u001b[33m\"seldonio/mab_epsilon_greedy:1.1\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"eg-router\"\u001b[39;49;00m\r\n",
      "\t\t\t}],\r\n",
      "\t\t\t\u001b[34;01m\"terminationGracePeriodSeconds\"\u001b[39;49;00m: \u001b[34m20\u001b[39;49;00m\r\n",
      "\t\t    }}\r\n",
      "\t        ],\r\n",
      "\t\t\u001b[34;01m\"graph\"\u001b[39;49;00m: {\r\n",
      "\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"eg-router\"\u001b[39;49;00m,\r\n",
      "\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m:\u001b[33m\"ROUTER\"\u001b[39;49;00m,\r\n",
      "\t\t    \u001b[34;01m\"parameters\"\u001b[39;49;00m: [\r\n",
      "\t\t\t{\r\n",
      "\t\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"n_branches\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"value\"\u001b[39;49;00m: \u001b[33m\"2\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m: \u001b[33m\"INT\"\u001b[39;49;00m\r\n",
      "\t\t\t},\r\n",
      "\t\t\t{\r\n",
      "\t\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"epsilon\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"value\"\u001b[39;49;00m: \u001b[33m\"0.2\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m: \u001b[33m\"FLOAT\"\u001b[39;49;00m\r\n",
      "\t\t\t},\r\n",
      "\t\t\t{\r\n",
      "\t\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"verbose\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"value\"\u001b[39;49;00m: \u001b[33m\"1\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m: \u001b[33m\"BOOL\"\u001b[39;49;00m\r\n",
      "\t\t\t}\r\n",
      "\t\t    ],\r\n",
      "\t\t    \u001b[34;01m\"children\"\u001b[39;49;00m: [\r\n",
      "\t\t\t{\r\n",
      "\t\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier-1\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"endpoint\"\u001b[39;49;00m:{\r\n",
      "\t\t\t\t\u001b[34;01m\"type\"\u001b[39;49;00m:\u001b[33m\"REST\"\u001b[39;49;00m\r\n",
      "\t\t\t    },\r\n",
      "\t\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m:\u001b[33m\"MODEL\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"children\"\u001b[39;49;00m:[]\r\n",
      "\t\t\t},\r\n",
      "\t\t\t{\r\n",
      "\t\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier-2\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"endpoint\"\u001b[39;49;00m:{\r\n",
      "\t\t\t\t\u001b[34;01m\"type\"\u001b[39;49;00m:\u001b[33m\"REST\"\u001b[39;49;00m\r\n",
      "\t\t\t    },\r\n",
      "\t\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m:\u001b[33m\"MODEL\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"children\"\u001b[39;49;00m:[]\r\n",
      "\t\t\t}   \r\n",
      "\t\t    ]\r\n",
      "\t\t}\r\n",
      "\t    }\r\n",
      "\t]\r\n",
      "    }\r\n",
      "}\r\n",
      "\t\t\r\n",
      "\r\n",
      " \r\n"
     ]
    }
   ],
   "source": [
    "!helm template ../../helm-charts/seldon-mab --name mymab --set oauth.key=oauth-key --set oauth.secret=oauth-secret | pygmentize -l json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waiting for deployment \"mymab-abtest-classifier-1-0\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"mymab-abtest-classifier-1-0\" successfully rolled out\n",
      "deployment \"mymab-abtest-classifier-2-1\" successfully rolled out\n",
      "Waiting for deployment \"mymab-abtest-eg-router-2\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymab-abtest-eg-router-2\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymab-abtest-eg-router-2\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymab-abtest-eg-router-2\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymab-abtest-eg-router-2\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymab-abtest-eg-router-2\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymab-abtest-eg-router-2\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"mymab-abtest-eg-router-2\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/mymab-abtest-classifier-1-0\n",
    "!kubectl rollout status deploy/mymab-abtest-classifier-2-1\n",
    "!kubectl rollout status deploy/mymab-abtest-eg-router-2 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### REST Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"4a9p29vil4nitor6vn18mmcgjv\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "      \"eg-router\": 0\n",
      "    },\n",
      "    \"requestPath\": {\n",
      "      \"eg-router\": \"seldonio/mab_epsilon_greedy:1.1\",\n",
      "      \"classifier-1\": \"seldonio/mock_classifier:1.0\"\n",
      "    }\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.09101330616093926]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "r = rest_request_ambassador(\"mymab\",API_AMBASSADOR)\n",
    "print(r.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"7892c1ce-ab4b-43b8-a836-fbf872d7dba9\",\"token_type\":\"bearer\",\"expires_in\":42796,\"scope\":\"read write\"}\n",
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"p8kobe9ap7uhols611194th32d\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "      \"eg-router\": 0\n",
      "    },\n",
      "    \"requestPath\": {\n",
      "      \"eg-router\": \"seldonio/mab_epsilon_greedy:1.1\",\n",
      "      \"classifier-1\": \"seldonio/mock_classifier:1.0\"\n",
      "    }\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.1002378266885396]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "r = rest_request_api_gateway(\"oauth-key\",\"oauth-secret\")\n",
    "print(r.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### gRPC Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "meta {\n",
       "  puid: \"vpomgkhv3v37vm0vg4sqip6i05\"\n",
       "  routing {\n",
       "    key: \"eg-router\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"classifier-1\"\n",
       "    value: \"seldonio/mock_classifier:1.0\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"eg-router\"\n",
       "    value: \"seldonio/mab_epsilon_greedy:1.1\"\n",
       "  }\n",
       "}\n",
       "data {\n",
       "  names: \"proba\"\n",
       "  tensor {\n",
       "    shape: 1\n",
       "    shape: 1\n",
       "    values: 0.07252885966272392\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grpc_request_ambassador(\"mymab\",API_AMBASSADOR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"7892c1ce-ab4b-43b8-a836-fbf872d7dba9\",\"token_type\":\"bearer\",\"expires_in\":42772,\"scope\":\"read write\"}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "meta {\n",
       "  puid: \"9grr0vjkscmg1a28pqc6tl34tl\"\n",
       "  routing {\n",
       "    key: \"eg-router\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"classifier-1\"\n",
       "    value: \"seldonio/mock_classifier:1.0\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"eg-router\"\n",
       "    value: \"seldonio/mab_epsilon_greedy:1.1\"\n",
       "  }\n",
       "}\n",
       "data {\n",
       "  names: \"proba\"\n",
       "  tensor {\n",
       "    shape: 1\n",
       "    shape: 1\n",
       "    values: 0.07922567094086788\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grpc_request_api_gateway(\"oauth-key\",\"oauth-secret\",grpc_endpoint=\"localhost:8004\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "release \"mymab\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!helm delete mymab --purge"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Serve Single Model with Outlier Detector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   mymodel\n",
      "LAST DEPLOYED: Fri Oct 19 18:52:39 2018\n",
      "NAMESPACE: seldon\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1alpha2/SeldonDeployment\n",
      "NAME     AGE\n",
      "mymodel  0s\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install ../../helm-charts/seldon-single-model --name mymodel --set outlier_detection.enabled=true --set oauth.key=oauth-key --set oauth.secret=oauth-secret"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\r\n",
      "\u001b[04m\u001b[31;01m#\u001b[39;49;00m \u001b[04m\u001b[31;01mS\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mu\u001b[39;49;00m\u001b[04m\u001b[31;01mr\u001b[39;49;00m\u001b[04m\u001b[31;01mc\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01m:\u001b[39;49;00m \u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mi\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\u001b[04m\u001b[31;01mg\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01m/\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01mp\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01m/\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01m.\u001b[39;49;00m\u001b[04m\u001b[31;01mj\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\r\n",
      "{\r\n",
      "    \u001b[34;01m\"apiVersion\"\u001b[39;49;00m: \u001b[33m\"machinelearning.seldon.io/v1alpha2\"\u001b[39;49;00m,\r\n",
      "    \u001b[34;01m\"kind\"\u001b[39;49;00m: \u001b[33m\"SeldonDeployment\"\u001b[39;49;00m,\r\n",
      "    \u001b[34;01m\"metadata\"\u001b[39;49;00m: {\r\n",
      "        \u001b[34;01m\"labels\"\u001b[39;49;00m: {\r\n",
      "            \u001b[34;01m\"app\"\u001b[39;49;00m: \u001b[33m\"seldon\"\u001b[39;49;00m\r\n",
      "        },\r\n",
      "        \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"mymodel\"\u001b[39;49;00m\r\n",
      "    },\r\n",
      "    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "        \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"mymodel\"\u001b[39;49;00m,        \r\n",
      "        \u001b[34;01m\"oauth_key\"\u001b[39;49;00m: \u001b[33m\"oauth-key\"\u001b[39;49;00m,\r\n",
      "        \u001b[34;01m\"oauth_secret\"\u001b[39;49;00m: \u001b[33m\"oauth-secret\"\u001b[39;49;00m,\r\n",
      "        \u001b[34;01m\"predictors\"\u001b[39;49;00m: [\r\n",
      "            {\r\n",
      "                \u001b[34;01m\"componentSpecs\"\u001b[39;49;00m: [{\r\n",
      "                    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "                        \u001b[34;01m\"containers\"\u001b[39;49;00m: [\r\n",
      "                            {\r\n",
      "                                \u001b[34;01m\"image\"\u001b[39;49;00m: \u001b[33m\"seldonio/mock_classifier:1.0\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"imagePullPolicy\"\u001b[39;49;00m: \u001b[33m\"IfNotPresent\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"resources\"\u001b[39;49;00m: {\r\n",
      "                                    \u001b[34;01m\"requests\"\u001b[39;49;00m: {\r\n",
      "                                        \u001b[34;01m\"memory\"\u001b[39;49;00m: \u001b[33m\"1Mi\"\u001b[39;49;00m\r\n",
      "                                    }\r\n",
      "                                }\r\n",
      "                            }\r\n",
      "                        ],\r\n",
      "                        \u001b[34;01m\"terminationGracePeriodSeconds\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m\r\n",
      "                    }}        \t\t\t\t   \r\n",
      "\t\t    ,\r\n",
      "\t\t    {\r\n",
      "\t\t    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\u001b[34;01m\"containers\"\u001b[39;49;00m: [\r\n",
      "\t\t\t    {\r\n",
      "\t\t\t\t\u001b[34;01m\"image\"\u001b[39;49;00m: \u001b[33m\"seldonio/outlier_mahalanobis:0.3\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"imagePullPolicy\"\u001b[39;49;00m: \u001b[33m\"IfNotPresent\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"outlier-detector\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"resources\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t    \u001b[34;01m\"requests\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t\t\u001b[34;01m\"memory\"\u001b[39;49;00m: \u001b[33m\"1Mi\"\u001b[39;49;00m\r\n",
      "\t\t\t\t    }\r\n",
      "\t\t\t\t}\r\n",
      "\t\t\t    }\r\n",
      "\t\t\t],\r\n",
      "\t\t\t\u001b[34;01m\"terminationGracePeriodSeconds\"\u001b[39;49;00m: \u001b[34m20\u001b[39;49;00m\r\n",
      "\t\t    }\r\n",
      "\t\t    }\t\t\t\t   \r\n",
      "\t        ],\r\n",
      "                \u001b[34;01m\"graph\"\u001b[39;49;00m:\r\n",
      "\t\t {\r\n",
      "\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"outlier-detector\"\u001b[39;49;00m,\r\n",
      "\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m: \u001b[33m\"TRANSFORMER\"\u001b[39;49;00m,\r\n",
      "\t\t    \u001b[34;01m\"parameters\"\u001b[39;49;00m: [\r\n",
      "\t\t\t{\r\n",
      "\t\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"n_components\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"value\"\u001b[39;49;00m: \u001b[33m\"2\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m: \u001b[33m\"INT\"\u001b[39;49;00m\r\n",
      "\t\t\t}\r\n",
      "\t\t    ],\r\n",
      "\t\t    \u001b[34;01m\"endpoint\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\u001b[34;01m\"type\"\u001b[39;49;00m: \u001b[33m\"REST\"\u001b[39;49;00m\r\n",
      "\t\t    },\r\n",
      "\t\t     \u001b[34;01m\"children\"\u001b[39;49;00m: [\t\t\t \r\n",
      "\t\t{\r\n",
      "                    \u001b[34;01m\"children\"\u001b[39;49;00m: [],\r\n",
      "                    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier\"\u001b[39;49;00m,\r\n",
      "                    \u001b[34;01m\"endpoint\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\u001b[34;01m\"type\"\u001b[39;49;00m : \u001b[33m\"REST\"\u001b[39;49;00m\r\n",
      "\t\t    },\r\n",
      "                    \u001b[34;01m\"type\"\u001b[39;49;00m: \u001b[33m\"MODEL\"\u001b[39;49;00m\r\n",
      "                }\r\n",
      "\t\t     ]},\t\t\t \t\t\t \r\n",
      "                \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"mymodel\"\u001b[39;49;00m,\r\n",
      "                \u001b[34;01m\"replicas\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m,\r\n",
      "\t\t\u001b[34;01m\"labels\"\u001b[39;49;00m: {\r\n",
      "\t\t    \u001b[34;01m\"version\"\u001b[39;49;00m : \u001b[33m\"v1\"\u001b[39;49;00m\r\n",
      "\t\t}\r\n",
      "            }\r\n",
      "        ]\r\n",
      "    }\r\n",
      "}\r\n"
     ]
    }
   ],
   "source": [
    "!helm template ../../helm-charts/seldon-single-model --name mymodel --set outlier_detection.enabled=true --set oauth.key=oauth-key --set oauth.secret=oauth-secret | pygmentize -l json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "deployment \"mymodel-mymodel-classifier-0\" successfully rolled out\n",
      "Waiting for deployment \"mymodel-mymodel-svc-orch\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"mymodel-mymodel-svc-orch\" successfully rolled out\n",
      "Waiting for deployment \"mymodel-mymodel-outlier-detector-1\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymodel-mymodel-outlier-detector-1\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymodel-mymodel-outlier-detector-1\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymodel-mymodel-outlier-detector-1\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymodel-mymodel-outlier-detector-1\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymodel-mymodel-outlier-detector-1\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymodel-mymodel-outlier-detector-1\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"mymodel-mymodel-outlier-detector-1\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/mymodel-mymodel-classifier-0\n",
    "!kubectl rollout status deploy/mymodel-mymodel-svc-orch\n",
    "!kubectl rollout status deploy/mymodel-mymodel-outlier-detector-1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### REST Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"a1m9eqjalqu2vtg2fcv24iutqv\",\n",
      "    \"tags\": {\n",
      "      \"outlierScore\": [0.0]\n",
      "    },\n",
      "    \"routing\": {\n",
      "      \"outlier-detector\": -1\n",
      "    },\n",
      "    \"requestPath\": {\n",
      "      \"outlier-detector\": \"seldonio/outlier_mahalanobis:0.3\",\n",
      "      \"classifier\": \"seldonio/mock_classifier:1.0\"\n",
      "    }\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.09249787999217929]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "r = rest_request_ambassador(\"mymodel\",API_AMBASSADOR)\n",
    "print(r.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"7892c1ce-ab4b-43b8-a836-fbf872d7dba9\",\"token_type\":\"bearer\",\"expires_in\":42667,\"scope\":\"read write\"}\n",
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"sb5c4vdclkoihsumghjmm8jkfo\",\n",
      "    \"tags\": {\n",
      "      \"outlierScore\": [0.0]\n",
      "    },\n",
      "    \"routing\": {\n",
      "      \"outlier-detector\": -1\n",
      "    },\n",
      "    \"requestPath\": {\n",
      "      \"outlier-detector\": \"seldonio/outlier_mahalanobis:0.3\",\n",
      "      \"classifier\": \"seldonio/mock_classifier:1.0\"\n",
      "    }\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.07854121999267133]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "r = rest_request_api_gateway(\"oauth-key\",\"oauth-secret\")\n",
    "print(r.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### gRPC Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "meta {\n",
       "  puid: \"hnr03ujhprah78b2lrbd418u59\"\n",
       "  tags {\n",
       "    key: \"outlierScore\"\n",
       "    value {\n",
       "      list_value {\n",
       "        values {\n",
       "          number_value: 0.0\n",
       "        }\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "  routing {\n",
       "    key: \"outlier-detector\"\n",
       "    value: -1\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"classifier\"\n",
       "    value: \"seldonio/mock_classifier:1.0\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"outlier-detector\"\n",
       "    value: \"seldonio/outlier_mahalanobis:0.3\"\n",
       "  }\n",
       "}\n",
       "data {\n",
       "  names: \"proba\"\n",
       "  tensor {\n",
       "    shape: 1\n",
       "    shape: 1\n",
       "    values: 0.0861506710600797\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grpc_request_ambassador(\"mymodel\",API_AMBASSADOR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"7892c1ce-ab4b-43b8-a836-fbf872d7dba9\",\"token_type\":\"bearer\",\"expires_in\":42650,\"scope\":\"read write\"}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "meta {\n",
       "  puid: \"e4aqe6s5ei8885o92804knc3n2\"\n",
       "  tags {\n",
       "    key: \"outlierScore\"\n",
       "    value {\n",
       "      list_value {\n",
       "        values {\n",
       "          number_value: 191.44918593397708\n",
       "        }\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "  routing {\n",
       "    key: \"outlier-detector\"\n",
       "    value: -1\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"classifier\"\n",
       "    value: \"seldonio/mock_classifier:1.0\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"outlier-detector\"\n",
       "    value: \"seldonio/outlier_mahalanobis:0.3\"\n",
       "  }\n",
       "}\n",
       "data {\n",
       "  names: \"proba\"\n",
       "  tensor {\n",
       "    shape: 1\n",
       "    shape: 1\n",
       "    values: 0.11002593453960169\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grpc_request_api_gateway(\"oauth-key\",\"oauth-secret\",grpc_endpoint=\"localhost:8004\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "release \"mymodel\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!helm delete mymodel --purge"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
